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EXECUTIVE  SUMMARY 


CH2M  performed  an  Environmental  Security  Technology  Certification  Program  (ESTCP) 
Munitions  Response  Live  Site  Demonstration  at  Munitions  Response  Site  (MRS)  R-04A  West,  at 
Tobyhanna  Anny  Depot  (TOAR)  Fonnerly  Used  Defense  Site  (FUDS),  in  Pennsylvania,  in  the 
summer  of  20 15. The  demonstration  involved  the  participation  of  two  advanced  electromagnetic 
induction  (EMI)  sensors  and  was  designed  to  investigate  the  use  of  these  advanced  EMI  sensors 
at  a  densely  wooded  site  with  challenging  microterrain  features  (e.g.,  impact  craters,  rocks, 
boulders,  gullies)  for  detecting  munitions  down  to  the  size  of  37-millimeter  projectiles.  The  Naval 
Research  Laboratory’s  Time-domain  Electromagnetic  Multisensor  Towed  Array  Detection 
System  2x2  (TEMTADS)  was  demonstrated  in  both  dynamic  and  cued  modes  in  a  two-person 
litter  carry  mode.  Collection  and  processing  of  dynamic  and  cued  data  with  a  man-portable  vector 
(MPV)  was  perfonned  by  Black  Tusk  Geophysics;  those  results  are  not  addressed  in  this  report. 
Both  instruments  were  coupled  with  a  Trimble  Robotic  Total  Station  for  positioning. 

Approximately  0.71  acre  was  dynamically  surveyed  with  the  TEMTADS  system.  Production  was 
significantly  hindered  by  the  remote  location  and  site  conditions,  and  only  one  of  the  four  grids 
initially  selected  for  investigation  achieved  100%  coverage  (not  including  gaps  due  to  physical 
obstructions).  Based  upon  this  effort,  production  rates  are  estimated  to  be  much  lower  than 
production  rates  would  be  under  less  challenging  conditions. 

A  total  of 429  anomalies  were  identified  by  the  TEMTADS  dynamic  data  analysis.  These  anomaly 
locations  and  an  additional  68  targets  selected  by  the  MPV  survey  were  interrogated  (cued  data 
collection)  and  classified  with  the  TEMTADS  system.  The  cued  data  were  used  to  classify  each 
target  as  being  a  potential  target  of  interest  (TOI)  (dig)  or  high-likelihood  non-TOI  (do  not  dig). 
The  TEMTADS  successfully  detected  and  classified  all  known  TOIs  (seeds  and  native  TOIs). 
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1.0  INTRODUCTION 


This  demonstration  report  details  the  Environmental  Security  Technology  Certification  Program 
(ESTCP)  Munitions  Response  Live  Site  Demonstration  at  Munitions  Response  Site  (MRS)  R-04A 
West,  at  Tobyhanna  Anny  Depot  (TOAR)  Fonnerly  Used  Defense  Site  (FUDS),  in  Pennsylvania, 
perfonned  in  the  summer  of  2015.  The  project  involved  the  demonstration  of  two  advanced 
electromagnetic  induction  (EMI)  sensors,  the  Time-domain  Electromagnetic  Multisensor  Towed 
Array  Detection  System  2x2  (TEMTADS)  and  the  man-portable  vector  (MPV)  systems,  both  of 
which  were  coupled  with  a  Trimble  Robotic  Total  Station  (RTS)  for  positioning.  The  MPV  data 
collection  and  processing  were  performed  by  Black  Tusk  Geophysics  and  are  not  addressed  in  this 
report.  The  demonstration  was  perfonned  in  accordance  with  the  Live  Site  Demonstrations 
TEMTADS  2x2  Demonstration  Plan  MRS-R04A  (West),  Tobyhanna  Army  Depot  FUDS, 
Pennsylvania,  ESTCP  Project  MR-201314  (ESTCP,  2015). 

1.1  BACKGROUND 

CH2M  HILL,  Inc.  (CH2M),  performed  this  work  for  ESTCP  under  contract  W912HQ-13-C-0039. 
The  demonstration  was  designed  to  investigate  the  use  of  advanced  EMI  sensors  at  a  densely 
wooded  site  with  challenging  microterrain  features  (e.g.,  impact  craters,  rocks,  boulders,  gullies) 
for  detection  of  37-millimeter  (mm)  projectiles.  As  a  result  of  the  site  conditions,  the  TEMTADS 
was  operated  in  two-person  litter  carry  mode  during  dynamic  (i.e.,  detection)  data  collection. 

1.2  OBJECTIVE  OF  THE  DEMONSTRATION 

The  overall  objective  was  to  demonstrate  the  detection  and  classification  performance  of  the 
sensors  in  challenging  conditions  for  targets  of  interest  (TOI)  as  small  as  37-mm  projectiles  and  to 
provide  data  for  comparing  the  TEMTADS  capabilities  under  these  conditions  to  those  of  the 
MPV.  (Note  that  this  report  covers  the  TEMTADS  operations  and  analysis  only;  comparison  of 
the  results  achieved  with  those  of  the  MPV  will  be  perfonned  by  the  ESTCP  Program  Office.) 

CH2M  perfonned  the  following  tasks: 

•  Collection  of  dynamic  transect  data  across  an  1 1-acre  parcel  using  a  Geometries  G-858G 
magnetometer 

•  Selection  of  demonstration  site  based  on  G-858G  data 

•  Reduction  of  vegetation 

•  Placement  of  subsurface  “blind”  seeds  within  the  demonstration  site 

•  Establishment  of  an  instrument  verification  strip  (IVS) 

•  Collection  of  dynamic  data  cueing  the  TEMTADS 

•  Interrogation  of  cued  targets  using  the  TEMTADS 

•  Processing  of  dynamic  and  cued  geophysical  data 

•  Reacquisition  and  intrusive  investigation  of  targets  selected  for  cued  interrogation 
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1.3  REGULATORY  DRIVERS 


The  Military  Munitions  Response  Program  (MMRP)  is  charged  with  characterizing  and,  where 
necessary,  remediating  MRSs.  When  an  MRS  is  remediated,  it  is  typically  mapped  with  a 
geophysical  system,  based  on  either  a  magnetometer  or  EMI  sensor,  and  the  locations  of  all 
detectable  signals  are  investigated.  Many  of  these  detections  do  not  correspond  to  munitions,  but 
rather  to  hannless  metallic  objects  or  geology;  field  experience  indicates  that  often  in  excess  of 
90%  of  objects  excavated  during  the  course  of  a  munitions  response  are  found  to  be  nonhazardous 
items.  Conventional  geophysical  technology,  as  it  is  traditionally  implemented,  does  not  provide 
a  physics-based,  quantitative,  validated  means  to  discriminate  between  hazardous  munitions  and 
nonhazardous  items. 

With  no  information  to  suggest  the  origin  of  the  signals,  the  sources  of  all  anomalies  are  currently 
treated  as  though  they  were  intact  munitions  when  they  are  dug  up.  They  are  carefully  excavated 
by  certified  unexploded  ordnance  (UXO)  technicians  using  a  process  that  often  requires  expensive 
safety  measures,  such  as  barriers  or  exclusion  zones.  As  a  result,  most  of  the  costs  to  remediate  a 
munitions-impacted  site  are  currently  spent  on  excavating  targets  that  pose  no  threat.  If  these  items 
could  be  detennined  with  high  confidence  to  be  nonhazardous,  some  of  these  expensive  measures 
could  be  eliminated  or  the  items  could  be  left  unexcavated  entirely. 

The  MMRP  is  severely  constrained  by  available  resources,  and  remediation  of  the  entire  inventory 
using  current  practices  is  cost  prohibitive  within  current  and  anticipated  funding  levels.  With 
current  planning,  estimated  completion  dates  for  munitions  response  on  many  sites  are  decades 
out.  The  United  States  Department  of  Defense’s  (DoD’s)  Defense  Science  Board  (DSB)  observed 
in  its  2003  report  that  significant  cost  savings  could  be  realized  if  successful  classification  between 
munitions  and  other  sources  of  anomalies  could  be  implemented.  If  these  savings  were  realized, 
the  limited  resources  of  the  MMRP  could  be  used  to  accelerate  the  remediation  of  MRSs  that  are 
currently  forecast  to  be  untouched  for  decades. 
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2.0  TECHNOLOGY 


The  technology  used  by  CH2M  for  this  project  consisted  of  a  TEMTADS  advanced 
electromagnetic  sensor,  coupled  with  an  RTS  positioning  system.  Data  collection  is  initiated  and 
monitored  using  the  data  acquisition  user  interface  provided  with  the  TEMTADS  sensor. 

2.1  TECHNOLOGY  Description 

2.1.1  TEMTADS  2x2 

The  TEMTADS  comprises  four  individual  sensor  elements  arranged  in  a  2-by-2  array.  Each 
element  consists  of  a  transmit  coil  and  a  three-axis  receiver  cube.  Each  cube  has  dimensions  of  8 
centimeters  (cm).  The  center-to-center  distance  is  40  cm,  yielding  an  array  80  cm  by  80  cm.  A 
TEMTADS  sensor  element  under  construction  and  the  sensor  array  (with  protective  cover 
removed)  are  shown  in  Figure  2-1. 


♦T 
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♦2 
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x  Sensor  Orientation 


Direction  ofTravel 


EM  Sensor 


Figure  2-1.  Single-element  TEMTADS  Three-axis  Receiver  and  Transmit  Coil  (Left)  and 

Sensor  Array  (Right) 

Decay  data  are  collected  with  a  500-kilohertz  sample  rate  until  25  milliseconds  (ms)  after  turn  off 
of  the  excitation  pulse.  This  results  in  a  raw  decay  of  12,500  points,  which  is  too  many  to  be  used 
practically.  These  raw  decay  measurements  are  grouped  into  122  logarithmically  spaced  “gates” 
with  center  times  ranging  from  25  microseconds  (ps)  to  24.35  ms  with  5  percent  widths  and  are 
saved  to  disk. 

The  TEMTADS  is  a  person-portable  system  (Figure  2-2).  The  array  structure  is  fabricated  from 
PVC  and  Garolite  fiberglass.  The  array  is  normally  deployed  on  a  set  of  wheels,  resulting  in  a 
sensor-to-ground  offset  of  approximately  18  cm.  For  this  demonstration  the  unit  was  deployed  in 
two-person  litter  configuration  on  skids  with  approximately  the  same  sensor  height  as  the  wheeled 
configuration.  The  transmitter  electronics  and  the  data  acquisition  computer  are  mounted  on  the 
operator  backpack,  an  RTS  prism  is  mounted  on  top  of  a  Garolite  fiberglass  pole,  and  an  inertial 
measurement  unit  (IMU)  is  mounted  above  the  array.  The  TEMTADS  can  be  operated  in  two 
modes:  dynamic  and  cued.  Data  collection  is  controlled  in  dynamic  mode  using  G&G  Science’s 
EM3D  application  suite.  In  cued  mode,  the  locations  of  previously  identified  anomalies  are  flagged 
and  surveyed  with  static  measurements  directly  over  the  flag  location. 
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Figure  2-2.  TEMTADS  2x2  System 

Custom  software  written  by  the  Naval  Research  Laboratory  (NRL)  is  used  to  control  the  cued  data 
acquisition.  Both  sets  of  software  are  accessed  through  a  remote  desktop  on  a  computer  tablet  by 
a  third  operator  accompanying  the  two  carriers. 

2.1.2  Positioning  System 

Because  the  reliable  use  of  a  global  positioning  system  (GPS)  was  not  possible  for  the  conditions 
at  the  site,  positional  data  were  recorded  using  a  Trimble  R7  RTS  with  an  MT1000  active  prism 
centered  over  the  TEMTADS  array.  A  pole  was  added  to  the  system  to  raise  the  height  of  the  prism 
above  the  heads  of  the  operators,  thus  allowing  the  operators  to  walk  without  blocking  the  prism 
from  the  RTS  base  station.  Positional  data  were  logged  at  a  nominal  rate  of  10  hertz.  The  pitch, 
roll,  and  yaw  of  the  cart  were  recorded  using  an  IMU  mounted  beneath  the  prism. 

2.1.3  Data  Acquisition  User  Interface 

Data  collection  is  controlled  using  a  tablet,  which  wirelessly  (IEEE  802.1  lg)  communicates  with 
the  data  acquisition  computer  on  the  operator  backpack.  The  tablet  operator  also  manages  field 
notes  and  team  orienteering  functions. 

Screenshots  of  the  tablet  user  interface  are  shown  in  Figure  2-3. 
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Figure  2-3.  Dynamic  Data  Acquisition  Interface  (Top) 
and  Cued  Data  Acquisition  Interface  (Bottom) 

2.2  Limitations  of  the  Technology 

Although  the  TEMTADS  was  designed  to  be  used  in  difficult  terrain  and  in  wooded  areas  that  its 
larger  predecessors  (e.g.,  MetalMapper)  could  not  access,  the  microterrain  features  at  TOAR 
FUDS  likely  would  result  in  increased  sensor  noise  levels  due  to  the  bouncing  and  shaking  of  the 
array  as  its  skids  made  contact  with  the  ground  surface,  rocks,  and  fallen  logs.  Irregular  heights 
due  to  its  being  lifted  over  objects  may  also  decrease  sensor  sensitivity.  In  addition,  imprecision 
in  the  positioning  data  may  result  from  increased  pitch  and  roll  movements  exacerbated  by 
increased  height  of  the  RTS  prism.  As  a  result,  one  of  the  objectives  of  this  demonstration  was  to 
determine  how  well  the  TEMTADS  performs  in  litter  carry  mode  in  order  to  assess  the  efficacy  of 
this  modality  under  the  expected  site  conditions. 

Another  serious  limitation  is  anomaly  density;  for  all  advanced  EMI  systems  there  is  a  limiting 
anomaly  density  above  which  the  response  of  individual  targets  cannot  be  separated.  Recent 
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developments  in  multisource  solvers  have  facilitated  improved  results  within  elevated  anomaly 
density  areas,  although  for  sites  with  areas  of  significant  anomaly  density  the  result  may  still  be 
saturated  response  areas  that  cannot  be  subjected  to  classification.  An  exploratory  G-858G 
(magnetometer)  survey  was  conducted  to  identifying  suitable  density  survey  areas  for  the 
demonstration. 

Densely  wooded  environments  are  challenging  for  positioning  the  TEMTADS  sensor.  The 
advantages  of  using  the  TEMTADS  as  a  dynamic  detector  rely  upon  precise  positioning  that 
cannot  be  achieved  using  fiducial  marks  and  interpolation.  Although  RTS  systems  can  be  used  in 
wooded  environments  where  the  line  of  sight  along  the  survey  transect  is  disrupted,  localized  gap- 
fill  surveying  requiring  multiple  base  station  setups  is  needed  to  achieve  100  percent  coverage 
along  the  transects. 
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3.0  PERFORMANCE  OBJECTIVES 


Performance  objectives  for  the  demonstration  were  designed  to  provide  a  basis  for  evaluating  the 
perfonnance  of  the  TEMTADS.  The  minimum  acceptable  criteria  were  the  thresholds  used  to 
detennine  that  the  system  was  working  properly.  Values  below  this  threshold  are  a  potential  cause 
for  rejection  of  the  data  and  require  a  root  cause  analysis/corrective  action  if  appropriate  (including 
potential  re-collection  of  data).  The  nominal  success  criteria  represent  the  expected  achievable 
threshold.  Failure  to  meet  these  thresholds  requires  a  discussion  in  the  project  report  but  are  not 
cause  for  rejection  of  the  data. 

To  avoid  repetition,  the  specific  perfonnance  objectives  for  the  demonstration  are  presented  along 
with  the  results  in  Section  6. 
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4.0  SITE  DESCRIPTION 


The  demonstration  site  is  located  within  MRS-R04A  (West)  at  TOAR  FUDS,  which  is  situated 
within  Pennsylvania  State  Game  Lands.  The  site  is  used  for  recreational  activities  such  as  camping, 
hiking,  fishing,  mountain  biking,  and  snowmobiling.  Parts  of  the  MRS  are  located  within  a 
designated  natural  area  open  only  to  passive  recreation  and  hunting. 

Munitions  Response  (MR)  actions  are  ongoing  within  the  MRS.  CH2M  is  not  involved  with  these 
actions. 

4.1  Site  Selection 

MRS-R04A  (West)  encompasses  approximately  250  acres  and  is  characterized  by  densely 
wooded,  uneven  terrain.  Figure  4-1  shows  photos  of  the  existing  site  conditions  taken  by  CH2M 
during  a  site  reconnaissance  visit  on  May  19  and  20,  2015.  Figure  4-2  shows  the  MRS  with  the 
operational  grid  system  used  by  the  MR  contractor  performing  work  at  the  site.  Each  grid  square 
measures  100  feet  by  100  feet.  This  figure  also  presents  an  enlarged  view  of  the  1  l-acre  portion  of 
the  MRS  that  had  not  yet  been  cleared  by  the  MR  contractor  and  from  within  which  up  to  2  acres 
was  selected  for  the  demonstration.  Based  on  historical  live-fire  training  conducted  on  artillery 
ranges  at  TOAR  FUDS,  and  the  results  of  a  previously  completed  remedial  investigation,  this  MRS 
encompasses  an  impact  area. 


Figure  4-1.  MRS-R04A  (West)  Representative  Site  Photos 
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Information  that  would  enable  estimation  of  anomaly  densities  for  the  MRS  had  not  been  provided; 
therefore,  CH2M  performed  a  digital  geophysical  mapping  survey  with  the  Geometries  G-858G 
cesium  vapor  gradiometer  along  transects  extending  through  the  accessible  portions  of  the 
approximately  11 -acre  area  shown  on  Figure  4-2.  The  intended  nominal  transect  spacing  was  6 
meters  (m);  however,  because  no  vegetation  reduction  was  perfonned  in  advance  of  the  G-858G 
survey,  transect  spacing  and  percent  coverage  was  variable  across  the  area.  The  objective  of  the 
G-858G  survey  was  to  assess  relative  anomaly  density  in  order  to  identify  candidate  grids  for  the 
TEMTADS  and  MPV  demonstration.  Positional  data  for  the  G-858G  survey  were  recorded  using 
a  Trimble  ProXRS  differential  GPS  system  with  an  intended  submeter  horizontal  accuracy.  A 
Geometries  G-856  proton  precession  magnetometer  was  used  as  a  stationary  base  station  to  record 
ambient  magnetic  field  fluctuations  throughout  each  day  of  G-858G  data  collection  in  order  to 
facilitate  evaluation  of  total  field  data  recorded  by  each  G-858G  sensor. 

Processing  and  target  selection  of  the  G-858G  transect  data  were  perfonned  using  Geosoft,  and 
Visual  Sample  Plan  (VSP)  (Battelle  Memorial  Institute,  2015)  was  used  to  obtain  estimates  of 
anomaly  density  within  the  11 -acre  area.  Transect  paths  and  calculated  anomaly  densities  are 
shown  on  Figure  4-3.  From  these  data,  two  grid  pairs  (78/46-79/46  and  82/47-83/47)  were 
selected  for  the  demonstration.  CH2M  identified  these  two  sets  of  grids  because  they  were 
sufficiently  far  apart  from  each  other  to  facilitate  concurrent  data  collection  with  the  TEMTADS 
and  MPV  without  the  risk  of  the  two  sensors  interfering  with  each  other. 
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4.2  Brief  Site  History 

TOAR  was  established  as  Camp  Sumerall  when  the  United  States  purchased  33  square  miles  of 
property  in  Monroe  County  in  1909.  The  facility  was  used  for  a  variety  of  missions  throughout  the 
years. 

The  site  was  first  used  for  machine  gun  and  artillery  training  during  1913.  The  Army  and  National 
Guard  used  the  facility  from  1913  until  1949  for  field  artillery  practice.  Camp  Sumerall  was  also 
used  as  a  training  area  for  tanks  from  July  through  October  1918.  The  ranges  were  the  only  areas 
in  Pennsylvania  where  live  cannon  fire  was  permitted  from  1919  to  1932.  During  this  timeframe, 
the  rounds  were  mainly  75-mm  French  artillery.  The  range  area  between  Warnertown  and  Route 
611  became  a  temporary  Headquarters  Explosives  Depot.  An  estimated  4  million  pounds  of  high 
explosives  was  stored  from  February  1919  through  October  1919.  Bunkers  were  constructed  in  the 
current  State  Game  Lands  127.  The  storage  designation  only  lasted  10  months. 


11 


4.3  Munitions  Contamination 


Suspected  munitions  within  MRS-R04A  (West)  include  primarily  75-inm  and  155-inm  high 
explosives  (HE)  and  shrapnel  projectiles;  however,  during  the  remedial  investigation  conducted 
within  MRS-R04A  (West),  37-mm  HE  projectiles  were  reportedly  recovered  along  with  75-mm 
and  155-mm  HE  projectiles. 

4.4  Site  Geodetic  Control  Information 

CH2M  subcontracted  Taylor  Wiseman  &  Taylor,  a  Professional  Land  Surveyor  (PLS)  licensed  in 
Pennsylvania,  to  establish  temporary  benchmarks  for  use  as  control  for  the  RTS  during  the 
demonstration.  CH2M  did  not  have  infonnation  on  current  benchmarks  that  may  be  onsite  and  in 
use  by  other  contractors  working  at  the  site.  New  temporary  benchmarks  were  established  to  a 
third-order  (1:10,000)  accuracy.  Universal  Transverse  Mercator  (UTM)  Zone  18  North  is  used  as 
the  projection  and  the  horizontal  datum  is  North  American  Datum  1983  (NAD83),  CONUS;  the 
vertical  datum  is  North  American  Vertical  Datum  1988  (NAVD88).  All  geodetic  measurements 
and  reported  information  are  in  units  of  meters.  Additional  locations  were  established  by  the  PLS 
throughout  the  survey  grids,  as  needed,  to  maintain  line  of  sight  for  the  RTS  during  the  TEMTADS 
and  MPV  surveys. 

4.5  Site  Preparation 

Vegetation  reduction  was  performed  in  the  grids  selected  to  be  surveyed  with  the  TEMTADS  and 
the  MPV.  A  brush  reduction  team  removed  vegetation  smaller  than  6  inches  (in.)  in  diameter.  Low 
branches  were  cleared  to  8  feet  above  ground  to  reduce  obstruction  of  the  RTS  prism;  large  logs 
and  fallen  timber  were  not  removed. 

CH2M  also  perfonned  a  surface  clearance  of  the  selected  demonstration  grids  prior  to  collection 
of  dynamic  TEMTADS  and  MPV  data.  The  surface  clearance  was  performed  with  the  objective 
of  leaving  no  more  than  five  pieces  of  exposed  or  partially  exposed  metallic  objects  exceeding 
2  in.  in  any  dimension  within  any  0.2-acre  area. 
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5.0 


TEST  DESIGN 


5.1  Calibration  Activities 

5.1.1  Navigation  Control 

Two  control  monuments  were  used  by  the  PLS  to  gain  control  of  the  site:  #7  MON  FP5  and  #3 1 
IRC  FP5  AZ.  New  control  stakes  were  set  for  the  work  area  grids  known  as  MRS-R04A  West, 
grids  78/46-79/46  and  82/47-83/47.  These  points  were  set  by  establishing  a  pair  of  real  time 
kinematic  (RTK)  GPS  control  points  in  the  open  (no  tree  canopy),  outside  of  the  grid  areas,  running 
a  conventional  survey  traverse  to  the  grids,  then  rough-checking  the  traverse  point  position  with 
GPS  under  the  heavy  tree  canopy.  Due  to  the  heavy  woods  and  terrain,  control  point  recover  sheets 
were  not  produced.  From  these  control  points  16  additional  control  points  were  established  within 
the  four  grids.  Their  labels  and  coordinates  are  provided  in  Table  5-1. 


Table  5-1.  Geodetic  Control  Locations 


ID 

Easting  (m) 
NAD83/UTM18  N 

Northing  (m) 
NAD83/UTM18  N 

Elevation 
NAVD88  (m) 

1000 

459500.8 

4558192.63 

557.831 

1001 

459543.96 

4558291.43 

559.57 

1002 

459576.81 

4558229.49 

557.356 

1003 

459631.14 

4558194.27 

554.828 

1004 

459736.93 

4558215.96 

557.885 

1010 

459612.62 

4558212.64 

555.64 

1011 

459642.42 

4558175.84 

554.351 

1012 

459668.17 

4558209.79 

555.821 

1013 

459637.54 

4558211.29 

555.246 

1014 

459731.49 

4558208.47 

558.007 

1015 

459750.17 

4558244.65 

557.887 

1016 

459791.55 

4558239.09 

560.63 

1017 

459779.14 

4558208.38 

560.294 

1018 

459728.9 

4558227 

557.984 

1019 

459609.2 

4558176 

554.539 

1020 

459731.4 

4558241 

557.627 

1021 

459761.3 

4558225 

558.549 

1022 

459760.8 

4558209 

558.837 
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To  set  up  the  RTS  system,  three  control  points  were  used:  a  base  location,  a  back-sight,  and  a 
check  point.  For  the  IVS,  point  #1017  was  used  as  the  base.  The  checkpoint  was  recorded  and 
submitted  as  part  of  the  daily  quality  control  (QC)  position  check. 

5.1.2  Seeding 

After  vegetation  reduction  and  prior  to  conducting  the  dynamic  TEMTADS  and  MPV  surveys, 
CH2M  seeded  the  selected  demonstration  grids.  CH2M  UXO  personnel  buried  20  seed  items 
consisting  of  industry  standard  objects  (ISOs).  ISOs  are  commonly  available  pipe  sections  that 
have  been  well-characterized  through  data  measurements  and  modeling.  A  combination  of  small 
schedule  80  ISOs  (diameter  =  1 .3 1 5  in.,  length  =  4  in.)  and  medium  schedule  40  ISOs  (diameter  = 
2.375  in.,  length  =  8  in.)  were  used  and  buried  between  1  and  17  cm.  The  PLS  recorded  the 
locations  of  the  blind  seeds  at  the  time  of  emplacement.  All  seeds  locations  were  blind  to  data 
collection  and  analysis  personnel. 

5.1.3  Instrument  Verification  Strip 

An  area  near  the  demonstration  grids  was  located  to  establish  an  IVS  for  daily  verification  of 
proper  sensor  operation.  An  initial  background  (i.e.,  preseeded)  survey  was  conducted  using  the 
TEMTADS  to  identify  a  suitable  location  for  the  IVS,  so  that  burial  of  seed  items  was  sufficiently 
far  from  existing  geophysical  anomaly  locations  (so  the  response  from  the  seed  items  were  not 
impacted  by  responses  from  metallic  objects  of  unknown  nature  in  the  subsurface).  A  schematic 
of  the  IVS  is  shown  as  Figure  5-1  and  details  of  seed  items  placed  in  the  IVS  are  listed  in  Table 
5-2. 


14 


Instrument  Verification  Strip 


TEMTADS  2x2 


MRS-R04A  (West) 


459760  459764  459768  459772  459776 


ID 

Easting 

Northing 
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Figure  5-1.  As-built  IVS 


Table  5-2.  Details  of  the  Instrument  Verification  Strip 


Item 

ID 

Description 

Easting  (m) 

Northing  (m) 

Depth 

(cm) 

Inclination 

Azimuth 

ISO-1 

Small  ISO80 

459771.40 

4558208.22 

15 

Horizontal 

Across  Track 

ISO-2 

Small  ISO80 

459769.86 

4558208.31 

10 

Horizontal 

Along  Track 

ISO-3 

Medium  1SO40 

459767.21 

4558208.35 

30 

Horizontal 

Across  Track 

ISO-4 

Small  ISO80 

459765.27 

4558208.27 

15 

Vertical 

Vertical 

ISO-5 

N/A 

459762.10 

4558208.44 

N/A 

Blank 

Blank 

ISO  =  Industry  Standard  Object 
N/A  =  not  applicable 
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5.1.4  Sensor  Calibration 


The  TEMTADS  sensor  was  calibrated  during  the  initial  commissioning  of  the  system  at  the  NRL 
facility  at  Blossom  Point,  Maryland.  The  system  calibration  was  verified  on  a  regular  basis  via 
function  tests  and  daily  IVS  surveys.  The  function  tests  involve  measurement  of  the  system 
response  to  an  ISO  placed  in  a  known  location  relative  to  the  sensor.  Function  tests  were  perfonned 
a  minimum  of  twice  per  day  to  verify  the  proper  operation  of  all  of  the  sensor  transmit  and  receive 
components.  Results  of  the  function  tests  and  IVS  surveys  are  provided  in  the  discussion  of  QC 
testing  in  Section  5.2.2  (Dynamic  Data  QC)  and  Section  5.2.5  (Cued  Data  QC). 

5.2  Data  Collection  Activities 

5.2.1  Dynamic  Data  Collection 

Dynamic  detection  surveys  were  performed  with  the  TEMTADS  over  the  course  of  1 1  survey 
days,  from  12  August  to  4  September  2015  (during  this  period  there  were  a  number  of  no-collection 
days  due  to  equipment  failures  as  well  as  weather  delays  and  non-worked  weekends).  There  were 
a  total  of  8  days  of  initial  data  collection  and  an  additional  3  days  of  effort  filling  gaps  in  coverage. 
The  positions  of  each  measurement  were  detennined  using  an  RTS  prism  mounted  at  the  center  of 
the  array,  coil  geometry  relative  to  the  RTS  prism,  and  the  platform  attitude  (pitch,  roll,  and  yaw) 
derived  from  the  IMU.  CH2M’s  field  team  used  ropes  and  flagging  to  perform  data  collection  at  a 
line  spacing  of  50  cm,  which  provided  an  overlap  coverage  of  30  cm  to  reduce  the  chance  of  data 
gaps.  Figures  5-2  and  5-3  show  mosaics  of  the  data  collected  at  78/46  and  79/46  (the  western)  and 
82/47  and  83/47  (the  eastern)  grid  sites,  respectively. 

Dynamic  data  collection  was  hindered  by  the  rough  terrain  (boulders,  swamp,  felled  trees,  and 
standing  trees)  and  the  need  to  reestablish  position  control  on  a  separate  control  point  once  the 
angle  from  the  RTS  “gun”  became  not  optimal  or  a  tree  blocked  too  much  of  the  “gun”  site. 

Due  to  site  access  time  constraints,  it  was  decided  with  the  input  from  the  ESTCP  Program  Office 
to  focus  on  a  100%  completion  of  one  grid.  Grid  82/47  (Figure  5-3)  had  gaps  identified  by  CH2M 
and  the  data  were  collected  until  only  physical  obstacles  (not  RTS  “gun”  shadows)  inhibited  data 
collection.  Because  it  took  an  additional  3  days  to  complete  the  gap  fill  on  one  grid  (%  of  the 
planned  survey  area),  by  extrapolation  it  is  estimated  that  an  additional  9  days  would  have  been 
required  to  fill  all  of  the  gaps  in  the  planned  survey  areas. 

Of  the  0.89-acre  footprint  cleared,  approximately  0.71  acre  of  dynamic  detection  data  were 
collected  with  the  TEMTADS.  CH2M  evaluated  the  dynamic  TEMTADS  data  and  selected  429 
targets  based  on  anomaly  selection  thresholds  derived  from  IVS  and  dynamic  test  data. 
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Figure  5-2.  Dynamic  TEMTADs  2x2  Survey  in  Grids  78/46  and  79/46  (Western) 
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Figure  5  3.  Dynamic  TEMTADS  2x2  Survey  in  Grids  82/47  and  83/47  (Eastern) 

5.2.2  Dynamic  Data  QC 

Throughout  the  course  of  the  dynamic  detection  survey,  the  TEMTADS  system  was  tested  at  the 
IVS  on  a  twice-daily  basis  to  verify  proper  system  operation.  In  order  to  measure  precision  of  the 
system,  ongoing  analysis  was  performed  on  the  IVS  detection  results,  with  each  successive  day’s 
results  compared  to  the  averaged  results  of  all  previous  IVS  surveys  for  detection  offset  and 
amplitude  response  of  each  seed  item. 

The  positions  were  derived  from  the  dynamic  monostatic,  Z-component  response  amplitude 
anomaly  peaks  using  Geosoft’s  automatic  peak  picking  algorithm.  Figure  5-4  (left)  presents  the 
position  offsets  (relative  to  the  ground  truth)  for  each  of  the  IVS  items.  Figure  5-4  (right)  presents 
the  position  offsets  from  average  position.  The  errors  were  outside  the  stated  objective  of  0.25  m 
(objectives  presented  in  Table  6-5). 
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TOAR  TEMTADS  2x2  Dynamic 
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Figure  5-3.  Dynamic  TEMTADs  2x2  IVS  Survey  Position  Results  (Initial  Processing) 

During  the  field  operations,  the  lack  of  positioning  accuracy  was  assumed  to  be  a  function  of  the 
RTS  operating  in  the  wooded  environment.  After  the  project  was  completed,  a  review  of  the 
processing  approach  revealed  that  the  data  processor  had  failed  to  account  for  the  extended  height 
of  the  RTS  prism  above  the  TEMTADS  array.  Because  of  this,  the  pitch  and  roll  positioning 
corrections  were  not  accurate,  and  the  accuracy  of  the  results  was  compromised.  The  data  were 
reprocessed  using  the  correct  positioning  approach,  and  the  results  are  provided  in  Figure  5-5. 
While  these  results  indicate  considerable  improvement,  there  remains  very  little  difference 
between  the  accuracy  (results  against  ground  truth)  and  the  precision  (variability  in  the  results). 
This  indicates  that  the  positioning  methodology  used  does  not  have  the  same  precision  as  a  typical 
GPS-enabled  approach.  However,  the  precision  in  the  results  is  comparable  to  that  of  a  GPS- 
enabled  EM6 1 -MK2  survey,  and  the  dig  radius  of  1  m  used  during  the  intrusive  investigation  was 
sufficient  to  mitigate  the  effects  of  these  increased  errors  on  locating  the  anomaly  sources. 
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Figure  5-4.  Dynamic  TEMTADs  2x2  IVS  Survey  Position  Results  (after  Reprocessing) 

In  addition  to  the  daily  IVS  measurements,  the  functionality  of  the  TEMTADS  sensor  was  assessed 
daily  using  a  system  “function  test”  whereby  the  system  response  was  challenged  by  placing  a 
small  schedule  80  ISO  on  the  top  of  the  array  housing.  The  function  test  results  are  presented  on 

Figure  5-6. 


Dynamic  Function  Test  Results  (Rx  Response) 


Figure  5-5.  Dynamic  TEMTADS  IVS  Survey  Sensor  Function  Results 
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5.2.3  Dynamic  Data  Processing/Target  Selection 
Data  Processing 

The  raw  data  files,  comprising  raw  sensor,  RTS,  and  IMU  data,  were  loaded  into  the  UX- Analyze 
(UXA)  advanced  analysis  software  environment.  The  RTS  and  IMU  data  were  merged  with  the 
sensor  data  to  provide  georeferenced  positions  for  each  TEMTADS  transmit/receive  (Tx/Rx) 
combination  measurement.  The  monostatic  Z-component  Rx  coil  measurements  were  used  as  the 
basis  for  amplitude  response  anomaly  detection. 

The  background  monostatic  Z-component  responses  were  removed  using  a  de-median  filter  in 
which  long  wavelength  signals  (due  to  spatially  stable  soil  response  and  sensor  “zero  level”  drift) 
were  modeled  and  removed  by  calculating  the  median  over  a  large  moving  window  and  subtracting 
the  median  from  the  unleveled  data.  The  final  (leveled)  data  for  each  grid  were  interpolated  to  grid 
nodes  evenly  spaced  at  5  cm  using  the  Geosoft  Oasis  minimum  curvature  gridding  routine.  The 
interpolated  Geosoft  grid  files  were  mosaicked  to  create  a  full-site  Geosoft  grid  file  from  which 
anomalies  were  selected. 

Dynamic  TEMTADS  Target  Selection 

Targets  were  selected  using  amplitude  response  detection  by  applying  Geosoft’s  peak  detection 
function  to  the  interpolated  monostatic,  Z-component  responses  at  time  gate  5  (0.137  ms).  This  is 
similar  to  conventional  EM61-MK2  target  detection,  with  the  only  difference  being  the  higher 
resolution  of  the  TEMTADS  sensor  due  to  the  smaller  Tx/Rx  coil  footprints  and  more  densely 
sampled  data  set. 

Detection  perfonnance  is  a  function  of  the  signal-to-noise  ratio  of  the  detection  method,  where 
signal  is  derived  as  the  peak  anomaly  response  and  the  noise  is  calculated  as  the  root  mean  square 
of  the  non-anomalous  responses.  Typically,  a  signal-to-noise  ratio  of  5  is  used  to  maximize 
detection  of  real  targets  while  minimizing  false  detections  due  to  noise  in  the  data.  The  site-specific 
noise  levels  were  estimated  by  calculating  the  standard  deviation  of  the  leveled  data  (assuming 
perfect  leveling,  root  mean  square  noise  is  equivalent  to  the  standard  deviation  of  the  signal)  in 
non-anomalous  regions.  Noise  levels  of  0.46  millivolts  per  ampere  (mV/A)  resulted  in  a  detection 
threshold  of  2.3  mV/A.  This  threshold  provides  a  maximum  reliable  detection  depth  (assuming 
worst  case  orientation)  of  0.3  m  for  small  ISOs  (small  ISOs  are  similar  in  response  to  37-mm 
projectiles).  Note  that  optimally  oriented  small  ISO’s  and  37-mm  projectiles  will  be  detected  at 
greater  depths.  Using  this  threshold,  anomalies  were  automatically  selected  from  the  gridded  data 
using  Geosoft’s  peak  detection  algorithm.  All  anomaly  selections  were  manually  reviewed  by  the 
processing  geophysicist,  and  manual  additions  or  deletions  were  performed  where  required. 

CH2M  selected  a  total  of  429  anomaly  locations  for  cued  interrogation.  There  were  165  selections 
in  the  western  grids  (Figure  5-7)  and  264  selections  in  the  eastern  grids  (Figure  5-8).  In  addition 
to  these,  68  MPV  targets  selected  by  Black  Tusk  Geophysics  were  added  to  the  cued  interrogation 
and  intrusive  investigation  lists.  These  targets  were  MPV  targets  located  in  the  100%  coverage 
area  (eastern  grids)  that  did  not  have  a  corresponding  TEMTADS  anomaly  location  (due  primarily 
to  the  fact  the  MPV  could  access  locations  that  the  TEMTADS  could  not). 
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Figure  5-6.  Dynamic  TEMTADS  Data  with  Locations  of  Anomalies  Selected  for  Cued  Data 
Collection,  Grids  78/46  and  79/46  (Western  Grids) 
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Figure  5-7.  Dynamic  TEMTADS  Data  with  Locations  of  Anomalies  Selected  for  Cued  Data 
Collection,  Grids  82/47  and  83/47  (Eastern  Grids) 

5.2.4  Cued  Data  Collection 

Cued  surveys  were  performed  with  the  TEMTADS  over  the  course  of  7  days,  from  4  September 
(Julian  day  247)  to  17  September  (Julian  day  260),  2015.  Data  were  recorded  electronically  as 
collected  on  the  TEMTADS  backpack  data  acquisition  computer  hard  drive.  The  collected  data 
were  copied  and  backed  up  daily  onto  removable  media  and  transferred  daily  to  the  data  analyst 
for  QC  analysis. 

All  of  the  429  targets  detected  by  the  TEMTADS  were  reacquired  for  cued  interrogation  with  the 
TEMTADS.  Cued  measurements  at  the  68  MPV  target  locations  that  were  not  selected  by  the 
TEMTADS  were  also  collected,  resulting  in  a  total  of  497  reacquired  targets. 

5.2.5  Cued  Data  QC 

The  QC  implemented  throughout  the  cued  data  collection  included  the  following: 
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•  IVS  measurements  before  and  after  each  day  of  production  measurements 

•  Function  tests  collected  whenever  background  measurements  were  collected 

•  Transmit  current  and  receiver  decay  monitoring 

•  Field  inversion  monitoring 

•  Re-collection  where  horizontal  target  location  was  offset  by  more  than  40  cm 

Throughout  the  course  of  the  cued  data  collection,  the  TEMTADS  system  was  tested  at  the  IVS 
on  a  twice  daily  basis  to  verify  sensor  functionality.  The  daily  IVS  measurements  were  inverted, 
and  the  extrinsic  parameters  (source  location)  and  intrinsic  parameters  (source  polarizabilities  [[3s]) 
results  were  monitored  and  recorded. 

These  results  for  the  source  locations  are  presented  on  Figures  5-8.  All  but  one  of  the  source 
position  results  obtained  were  within  the  measurement  quality  objective  (MQO)of  25  cm  for 
accuracy  (position  relative  to  the  ground  truth)  and  all  of  the  results  were  within  the  20-cm  MQO 
for  precision  (offset  from  the  average  position).  Because  the  accuracy  results  for  ISO  1  exhibit  a 
bias,  and  this  bias  is  not  present  in  the  remaining  ISOs,  it  is  assumed  that  the  MQO  failure  is 
attributable  in  part  to  inaccuracy  in  the  ground  truth  for  ISO  1. 


Figure  5-8.  TEMTADS  Cued  Data  IVS  Dipole  Fit  Position  Results:  (Left)  Offset  from 
Ground  Truth  and  (Right)  Offset  from  Average  Position 

The  derived  source  [3s  were  assessed  by  perfonning  a  library  match  to  derive  decision  metrics 
(described  below  in  section  5.3.1)  for  each  measurement  and  the  results  are  presented  on  Figure 
5-9.  All  measurements  resulted  in  very  good  fits  to  their  respective  library  entries,  indicating 
proper  operation  of  the  system. 
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Figure  5-9.  TEMTADS  Cued  IVS  Decision  Metric  (Library  Match)  Results 

In  addition  to  the  twice-daily  IVS,  in-field  sensor  functionality  tests  were  performed  throughout 
each  survey  day  to  confirm  that  the  TEMTADS  system  components  were  functioning  within 
project  specifications.  Sensor  function  tests  were  performed  during  each  background  data 
collection  event.  The  sensor  function  test  results  are  shown  on  Figures  5-10,  5-11,  and  5-12. 
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Figure  5-10.  TEMTADS  Cued  Data  Sensor  Function  Test  Results  Rx  Response 
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Figure  5-11.  TEMTADS  Cued  Data  Sensor  Function  Test  Results  Rx  Response  Variation 


Figure  5-12.  TEMTADS  Cued  Data  Sensor  Function  Test  Results  Tx  Current 
5.3  Classification 

The  data  processing,  analysis,  and  classification  steps  undertaken  to  generate  a  dig/no-dig  decision 
for  each  target  are  described  below.  These  steps  were  perfonned  for  the  TEMTADS  data  using  the 
UXA  module  within  the  Geosoft  Oasis  Montaj  processing  platform. 

Cued  data  were  imported  into  the  Geosoft  UXA  Advanced  module  for  data  QC  and  inversion 
modeling.  The  data  were  levelled  using  background  data  collected  at  frequent  time  intervals  over 
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nearby,  anomaly-free  background  locations.  The  measurements  used  for  background  correction 
were  reviewed  for  variability  and  to  identify  any  outliers  which  may  correspond  to  measurements 
over  subsurface  metal.  To  minimize  errors  in  the  background  removal  process,  spatial  and 
temporal  distance  between  the  background  and  target  measurements  were  minimized. 

Target  data  were  inverted  using  both  single-source  and  multisource  dipole  response  models  to 
estimate  target  parameters.  The  principle  parameters  of  interest  for  use  in  classification  of  the 
targets  were  the  three  polarizabilities  (pi ,  P2,  and  P3)  estimated  for  each  target  by  UXA.  In 
addition  to  estimates  for  the  three  Ps  for  each  target,  an  estimated  location  and  depth  and  fit 
coherence  (i.e.,  the  correlation  between  the  observed  responses  and  the  model  predictions)  was 
also  returned  by  UXA  for  each  target  during  inversion. 

Classification  of  each  target  was  perfonned  using  the  intrinsic  features  (Ps)  derived  from  the 
single-source  and  multisource  inversion  processes.  Classification  was  based  primarily  upon  how 
well  the  derived  Ps  matched  the  library  of  candidate  TOI  types.  The  final  composition  of  the  library 
was  infonned  by  a  set  of  analyst  calibration  digs  (ACDs)  (described  in  detail  in  section  5.3.2). 

5.3.1  Library  Match  Decision  Metric 

Classification  is  based  primarily  on  the  goodness-of-fit  metric  (values  from  0.0  to  1.0)  generated 
by  UXA  during  a  comparison  of  the  P  values  estimated  for  each  surveyed  target  and  the  P  values 
in  the  site-specific  library  of  candidate  munitions.  This  comparison  is  performed  via  the  library 
match  utility  in  UXA.  The  goodness-of-fit  metric  is  a  measure  of  the  fit  correlation  between  a 
target  and  the  library  entry  that  best  fits  that  target,  with  higher  values  indicating  a  better  fit 
between  the  target  and  the  corresponding  item  in  the  library.  The  library  fit  analysis  matches  the 
following  four  combinations  of  Ps  to  those  of  the  candidate  library  TOIs: 

•  pi,  pl/p2,  pl/p3 

•  PkPl/p2 

•  pl/p2,  pl/p3 

•  PI 

The  confidence  metrics  for  each  fit  combination  are  averaged  to  derive  a  “decision  metric.”  This 
library  matching  process  is  performed  for  each  single-solver  model  and  every  target  in  each  of  the 
multisource  solver  candidate  realization  models.  For  each  flag  position,  the  best  library  fit  from 
the  single-solver  and  multisolver  targets  is  used  as  the  decision  metric,  which  is  subsequently  used 
to  rank  and  classify  the  target  list.  Values  below  the  analyst  threshold  (nominally  0.825)  are 
considered  non-TOI.  The  analyst  threshold  was  refined  using  results  from  a  set  of  ACDs. 

5.3.2  Analyst  Calibration  Digs 

Because  all  identified  anomalies  were  intrusively  investigated,  ACDs  were  not  performed 
separately  from  the  intrusive  investigation  phase.  Instead,  the  ACDs  were  simulated  through 
requests  for  ground  truth  from  the  ESTCP  program  office.  The  following  sections  describe  the 
rationale  for  the  ACD  selections. 
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Cluster  Analysis 

A  “cluster  analysis”  designed  to  identify  signatures  that  are  ubiquitous  to  the  site  was  performed 
using  the  UXA  signature  matching/cluster  identification  routines.  For  identified  clusters  that  were 
not  already  represented  in  the  library,  representative  samples  were  selected  for  addition  to  the  set 
of  ACDs  (in  this  case,  “selections”  provided  on  request  by  the  ESTCP  Program  Office  to  simulate 
the  ACD  process)  to  confirm  that  they  are  not  due  to  an  unexpected  TOI. 

Feature  Space  Analysis 

In  addition  to  the  library  match  decision  metric  described  above,  a  feature  space  analysis  was  also 
perfonned  to  identify  any  targets  that  did  not  match  a  specific  library  entry  but  had  the  combined 
characteristics  of  being  large,  rotationally  symmetric,  and  thick-walled.  Targets  identified  with 
these  characteristics  were  selected  and  added  to  the  ACD  selection  list. 

Dig/No-dig  Threshold  Calibration 

The  ACDs  were  also  used  to  finalize  the  analyst  threshold  (i.e.,  the  decision  metric  cutoff  value 
separating  the  prioritized  list  into  dig/no-dig  classifications).  These  targets  were  selected  by 
sampling  each  library  match  munition  type  above  and  below  the  initial  starting  metric  of  0.825. 
Because  the  analyst  threshold  must  be  set  low  enough  to  identify  TOIs  that  have  noisy 
polarizabilities  as  TOIs,  where  possible  targets  that  looked  qualitatively  like  a  TOI  were 
preferentially  selected.  Although  final  classification  is  based  upon  objective  numeric  criteria, 
qualitative  selection  of  these  threshold  calibration  digs  is  required  to  preferentially  select  those 
targets  that  are  likely  to  be  TOIs,  thus  calibrate  the  threshold  appropriately. 

Library  Entry  Verification 

The  initial  library  contained  a  comprehensive  list  of  munitions  including  entries  that  were  not  in 
the  list  of  munitions  expected  to  be  onsite  but  were  conservatively  left  in  as  a  representative 
size/shape  sample.  Where  one  or  more  of  these  entries  resulted  in  a  significant  number  of  matches, 
they  were  sampled  to  determine  if  they  were  actual  TOIs.  Where  these  samples  did  not  result  in 
the  recovery  of  a  TOI,  these  entries  were  removed  from  the  final  site-specific  list.  Selection  of 
targets  with  noise-free  polarizabilities  does  not  inform  the  threshold  calibration  because  if  they  are 
near  the  threshold  they  will  not  be  TOIs. 

5.3.3  Candidate  TOI  library 

The  initial  candidate  library  consisted  of  a  comprehensive  range  of  munitions  that  are  delivered 
with  the  UXA  installation  files  (listed  on  the  left  side  of  Figure  5-13).  The  final  site-specific  library 
of  candidate  TOIs  comprised  munitions  expected  to  be  onsite,  as  confirmed  by  the  results  of  the 
ACDs.  The  site-specific  library  had  one  or  more  entries  for  each  item  listed  on  the  right-hand  side 

of  Figure  5-13. 
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Comprehensive  Library  (all  UXO) 

Type 

Size 

Decay 

Symmetry 

Small  ISO 

0.21 

0.031 

1.02 

rifle  grenade 

0.79 

0.009 

1.03 

medium  ISO 

0.98 

0.046 

1.06 

large  ISO 

1.54 

0.054 

1.02 

81mm 

1.36 

0.032 

1.05 

75mm 

1.08 

0.057 

1.01 

60mm 

0.79 

0.038 

1.01 

Sin  proj 

1.70 

0.058 

1.05 

57mm 

0.68 

0.056 

1.02 

40mm  grenade 

0.28 

0.092 

1.01 

3in  stokes  mortar 

1.38 

0.070 

1.15 

3in  proj 

1.14 

0.052 

1.02 

37mm 

-0.08 

0.092 

1 

2.36in  rocket 

1.17 

0.075 

1.19 

155mm  proj 

2.10 

0.050 

1.13 

105  mm  proj 

1.59 

0.059 

1.14 

20mm 

-0.30 

0.022 

0.98 

MK2  Hand  grenade 

0.32 

0.001 

1.04 

Fuze  Piece/part/component 

-0.45 

0.009 

1.1 

TOAR  Library  Entries 

Munition 

Size 

Decay 

Symmetry 

Small  ISO 

0.21 

0.030 

1.02 

Medium  ISO 

0.98 

0.050 

1.06 

Large  ISO 

1.54 

0.050 

1.02 

37-mm  projectile 

0.20 

0.060 

1.00 

75-mm  projectile 

1.08 

0.057 

1.01 

155-mm  projectile 

2.10 

0.050 

1.13 

Figure  5-13.  Initial  (left)  and  Final,  Site-specific  TOI  Library  (Right)  for  TOAR 
5.4  Intrusive  Activity  and  Procedures 

CH2M  performed  intrusive  operations  from  13  September  through  1  October  20 15.  All  prosecuted 
targets  were  investigated  and  documented  according  to  the  procedures  outlined  below. 

5.4.1  Intrusive  Investigation  Procedures 

•  Reacquisition  of  targets.  Targets  were  previously  reacquired  by  the  PLS  for  the  cued 
interrogation  using  RTS.  The  pin  flag  was  located  approximately  50  cm  north  of  the  paint 
mark  denoting  the  target  location. 

•  Intrusively  investigate  the  anomaly.  Anomalies  were  excavated  to  30  cm  below  the 
expected  depth  below  ground  surface,  within  a  50-cm  radius  from  the  marked-out 
anomaly  location. 

•  Identify  recovered  item.  All  items  recovered  were  inspected  by  the  Unexploded 
Ordnance  Safety  Officer  and  Senior  UXO  Supervisor  to  ensure  that  each  item  was 
properly  identified  and  documented. 

•  Munitions  Response  Site  Information  Management  System  (MRSIMS)  data 
entry/whiteboard  and  photo.  Field  observations  of  each  recovered  item  were  entered 
into  CH2M’s  MRSIMS  field  tablets.  The  exact  location  and  depth  of  each  item  was 
recorded  by  a  CFI2M  field  geophysicist  using  the  RTS.  Required  information  was  written 
onto  a  whiteboard,  and  a  photo  was  taken  with  the  item. 

•  Postdig  clearance.  Before  declaring  a  dig  complete,  each  area  was  swept  with  a 
Schonstedt  magnetometer  and  a  White’s  All  Metal  detector  to  determine  if  any  additional 
items  remained. 
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•  QC  check.  Approximately  10  percent  of  all  excavations  were  checked  by  UXO  Quality 
Control  personnel  with  the  Schonstedt  magnetometer  and  a  White’s  All  Metal  detector  to 
ensure  the  hole  was  clear. 

•  Backfill  hole.  Once  the  excavation  was  declared  clear,  the  hole  was  backfilled  to  grade. 

5.4.2  Intrusive  Investigation  Results 

All  of  the  TEMTADS  cued  targets  were  intrusively  investigated  as  well  as  a  set  of  MPV  targets 
located  in  the  100%  coverage  area.  Time  constraints  imposed  by  site  access  limitations  did  not 
allow  for  all  of  the  MPV  targets  outside  the  100%  coverage  area  to  be  investigated.  During  the 
intrusive  investigation,  all  of  the  QC  seeds  were  located  within  the  0.4-m  MQOs  and  recovered. 
The  results  of  the  intrusive  investigations  were  provided  to  the  ESTCP  Program  Office  and  are  not 
presented  in  this  report. 

5.5  Deliverables 

The  following  deliverables  resulted  from  the  data  collection  at  TOAR: 

1 .  Dynamic  detection  data:  Raw  and  processed  dynamic  detection  data  were  provided  to 
the  ESTCP  Program  Office,  along  with  a  final  target  list  based  on  the  established  detection 
threshold 

2.  Cued  data:  Raw  sensor  data  (*.tem)  and  associated  GPS/IMU  data  (*.gps) 

3.  Cross-reference  list:  A  text  readable  table  that  associates  TEMTADS  filenames  with  each 
Target  ID,  and  provides  any  applicable  collection  notes 

4.  Intrusive  results:  The  intrusive  results  were  provided  to  the  ESTCP  Program  Office  in  the 
form  of  a  table  listing  the  results  for  each  location  identified  for  intrusive  investigation  as 
well  as  photos  detailing  the  metallic  items  that  were  removed  from  each  location.  Note  that 
all  of  the  intrusive  results  were  firewalled  from  the  data  analyst  until  after  the  analysis  was 
completed  (with  the  exception  of  the  ACD  selections  described  above) 

6.0  PERFORMANCE  ASSESSMENT 

The  perfonnance  objectives  for  this  classification  survey  and  the  corresponding  results  are 
summarized  in  Table  6-1.  Details  on  the  results  for  each  objective  are  subsequently  discussed  in 
the  following  sections. 
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TABLE  6-1.  Performance  Objectives  and  Results  for  this  Demonstration 


Performance 

Objective 

Metric 

Data  Required 

Minimum  Acceptable 
Criterion 

Result 

Dynamic 
survey  spatial 
coverage 

Effective 

footprint 

coverage 

Mapped  survey 
data 

100%  at  <  75  cm  cross¬ 
track  measurement  spacing 
with  intended  spacing  of 

50  cm 

Fail  (pass  for  100% 
coverage  area) 

Along-line 

measurement 

spacing 

Point  to  point 
sample  distance 

Mapped  survey 
data 

98%  <  25  cm;  no  gaps  >  40 
cm  unless  obstruction  or 
hazard  is  present 

Fail  (pass  for  100% 
coverage  area) 

Detection  of  all 
TOIs 

Percent  of  seed 
items  detected 

Seed  item 
locations 

Georeferenced 
anomaly  list 

100%  of  seed  items  within 
a  40-cm  radius  of  ground 
truth 

Fail  (all  seed  items  were 
detected  but  two  were  not 
within  the  specified 
distance  due  to  gaps  in 
coverage) 

Initial  dynamic 
survey  data 
positioning 

Accuracy  of 
derived  target 
positions 

Derived  target 
positions  from 
initial 

measurements  at 

the  IVS 

Derived  positions  within 

25  cm  of  the  ground  truth 

Fail  (one  outlier  from 
horizontal  targets-at  3 1 
cm) 

Ongoing 
dynamic  survey 
data  positioning 

Precision  of 
derived  target 
positions 

Derived  target 
positions  from 
daily 

measurements  at 

the  IVS 

Derived  positions  within 

25  cm  of  the  average 
positions  during  ongoing 
daily  measurements 

Fail  (two  outliers  from 
horizontal  targets — all 
within  30  cm) 

Initial  cued 
survey  data 
positioning 

Accuracy  of 
dipole-fit- 
derived  target 
positions 

Target  fit 
positions  from 
initial 

measurements  at 

the  IVS 

IVS  item  fit  locations 
within  25  cm  of  ground 
truth  locations 

Pass 

Ongoing  cued 
survey  data 
positioning 

Precision  of 
dipole-fit- 
derived  target 
positions 

Target  fit 
positions  from 
daily 

measurements  at 

the  IVS 

IVS  item  fit  locations 
within  ±20  cm  of  average 
fit  locations  during 
ongoing  daily 
measurements 

Pass 

Initial  cued 

sensor 

polarizability 

accuracy 

Accuracy  of 
dipole-fit- 
derived  intrinsic 
target  features 

Dipole-fit-derived 
polarizabilities 
from  initial 

measurements  at 

the  IVS 

Library  match  metric  >0.9 
to  initial  polarizabilities  for 
each  set  of  inverted 
polarizabilities 

Pass 

Ongoing  cued 
sensor 

polarizability 

precision 

Precision  of 
dipole-fit- 
derived  intrinsic 
target  features 

Dipole-fit-derived 
polarizabilities 
from  daily 
measurements  at 

the  IVS 

Match  metric  >0.95  to 
initial  polarizabilities  at  the 
IVS  for  each  set  of 
inverted  polarizabilities 
from  daily  measurements 

Pass 
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TABLE  6-1.  Performance  Objectives  and  Results  for  this  Demonstration 


Performance 

Objective 

Metric 

Data  Required 

Minimum  Acceptable 
Criterion 

Result 

Cued 

interrogation 

anomaly 

coverage 

Instrument 

position 

Cued  data 

100%  of  anomalies  where 
the  center  of  the  array  is 
positioned  within  30  cm  of 
anomaly  location 

Pass 

Correct 

classification  of 
TOIs 

Number  of  TOIs 

correctly 

identified 

Ranked  anomaly 
lists 

Scoring  reports 
from  ESTCP 
Program  Office 

100%  of  all  seed  targets 

100%  of  all  TOIs 
categorized  as  “dig”  or 
“cannot  analyze” 

Pass 

Model  results 
support 
classification 
decision 

Number  of 
anomalies 
classified  as 
“cannot 
analyze” 

Modeling  fit 
coherence  results 

>90%  of  targets  have  fit 
coherence  >  0.80 

Pass 

6.1  Dynamic  Survey  Spatial  Coverage 

The  TEMTADS  dynamic  detection  survey  was  designed  to  provide  100  percent  coverage  of  the 
investigation  area.  A  planned  transect  spacing  of  50  cm  was  used  to  ensure  sufficient  overlap  of 
the  80-cm  sensor  swath  footprint  between  traverses.  Data  were  collected  continuously  with  an 
even  walking  pace;  this  consistent  pace  allowed  the  RTS  base  to  reestablish  contact  with  the  RTS 
prism  on  the  TEMTADS  after  it  emerged  from  behind  an  obstacle.  Using  this  approach,  two  types 
of  gaps  exist  in  this  data  set:  one  caused  by  the  RTS  “gun”  shadow  created  when  the  TEMTADS 
is  carried  behind  a  tree,  and  one  created  by  obstacles  such  as  downed  trees,  upright  trees,  and 
boulders.  For  the  case  of  the  RTS  shadow,  data  were  not  interpolated  even  though  the  dynamic 
TEMTADS  data  were  still  being  collected  and  there  were  positioning  data  on  either  side  of  the 
tree,  the  positioning  data  were  not  interpolated  in  processing.  As  in  any  survey,  obstacles  such  as 
trees  and  boulders  precluded  coverage.  To  achieve  the  stated  goal  of  100%  coverage,  gaps  due  to 
interruption  of  the  RTS  data  required  re-collection,  whereas  gaps  due  to  physical  obstructions  were 
documented  as  such. 

Due  to  time  constraints,  the  gap  fills  required  to  achieve  the  100%  coverage  goal  were  perfonned 
on  only  one  of  the  four  grids  (grid  82/47).  This  grid  had  a  coverage  of  81.2%  prior  to  gap  fill  and 
94.1%  after  gaps  due  to  RTS  coverage  were  eliminated.  The  remaining  5.9%  was  due  to  physical 
obstacles;  thus  the  100%  coverage  metric  was  achieved  for  this  grid.  Grids  78/46  and  79/46 
(western  grids)  combined  had  an  overall  coverage  of  72.4%  with  no  gap  re-collection.  The  gap 
percentage  due  to  obstacles  was  not  calculated  because  the  locations  of  these  objects  were  not 
recorded.  Grids  82/47  and  83/47  (eastern  grids)  combined  had  7 1 .2%  coverage,  not  accounting  for 
obstacles. 
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6.2  Along-line  Measurement  Spacing 


The  TEMTADS  dynamic  detection  survey  was  designed  to  have  an  along-line  measurement 
spacing  of  15  cm  or  less  so  as  to  provide  a  dense  detection  dataset.  The  success  criterion  for  this 
objective  was  for  98%  of  mapped  data  points  to  be  within  25  cm  of  the  along-line  neighboring 
data  point.  This  objective  was  not  achieved,  as  gaps  created  by  the  RTS  “gun”  shadow  were  not 
interpolated  across  these  gaps.  The  heavily  wooded  site  inhibited  the  continuous  contact  between 
the  base  RTS  and  the  prism  located  on  the  TEMTADS. 

6.3  Detection  of  All  TOI 

This  objective  involved  the  detection  of  TOIs  during  the  dynamic  detection  survey  using  the  blind 
seeding  program.  Prior  to  the  dynamic  detection  survey,  20  seed  items  were  buried  within  the 
investigation  area.  These  seed  items  consisted  of  small  schedule  80  ISOs  and  medium  schedule  40 
ISOs  buried  at  depths  up  to  17  cm.  The  minimum  acceptable  criterion  of  100  percent  of  seed  items’ 
locations  being  predicted  within  a  40-cm  radius  of  ground  truth  was  not  met  (Table  6-2).  All  of 
the  seed  items  were  detected  within  the  40-cm  radius  from  the  recorded  PLS  position  with  the 
exception  of  Seeds  1  and  4.  The  detection  distance  for  these  seeds  was  outside  of  the  MQO  due  to 
the  gaps  in  coverage  discussed  in  Section  6.1.  All  of  the  seeds  in  the  100%  detection  grid  passed 
the  40-cm  MQO. 


Table  6-2.  TEMTADS  QC  Seed  Detection 


Seed  ID 

Delta  X  (m) 

Delta  Y  (m) 

Distance  (m) 

1 

-0.56 

0.00 

0.56 a 

2 

0.03 

0.21 

0.22 

3 

-0.13 

-0.20 

0.24 

4 

-0.17 

-0.39 

0.43 a 

5 

0.34 

-0.10 

0.35 

6 

0.08 

0.03 

0.09 

7 

-0.11 

0.05 

0.13 

8 

0.12 

-0.16 

0.20 

9 

0.13 

-0.06 

0.14 

10 

-0.31 

-0.06 

0.32 

11 

0.10 

-0.28 

0.30 

12 

0.16 

0.28 

0.32 

13 

0.18 

-0.01 

0.18 

14 

-0.05 

-0.06 

0.08 

15 

0.07 

-0.13 

0.14 

16 

-0.24 

-0.02 

0.24 
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Table  6-2.  TEMTADS  QC  Seed  Detection 


Seed  ID 

Delta  X  (m) 

Delta  Y  (m) 

Distance  (m) 

17 

0.13 

-0.11 

0.17 

18 

0.08 

0.16 

0.18 

19 

0.08 

-0.06 

0.10 

20 

0.29 

0.12 

0.32 

a  Outside  the  MQO. 


6.4  Initial  Dynamic  IVS  Survey  Data  Positioning 

This  objective  was  to  demonstrate  positioning  accuracy  during  initial  dynamic  data  collection  over 
the  IVS  with  the  TEMTADS  by  deriving  positions  within  25  cm  of  ground  truth.  The  metric  was 
assessed  by  evaluating  the  derived  target  positions  from  the  initial  data  collection  at  the  IVS  as 
determined  by  the  amplitude  response  anomaly  peaks  associated  with  the  seeded  items.  Results 
for  the  dynamic  data  responses  are  provided  in  Table  6-3.  The  largest  deviation  was  39  cm  for 
ISO-1,  which  means  that  this  objective  was  not  achieved  for  all  ISOs. 


Table  6-3.  TEMTADS  Initial  Dynamic  IVS  Survey  Positioning  Results 


Seed  Item 

Error (m) 

ISO-1 

0.39 

ISO-2 

0.25 

ISO-3 

0.25 

ISO-4 

0.08 

6.5  Ongoing  Dynamic  Survey  Data  Positioning 

This  objective  involved  the  repeatability  of  sensor  response  amplitude  over  the  course  of  the 
project  for  each  seed  item  buried  at  the  IVS.  The  minimum  success  criterion  was  that  the  derived 
positions  of  the  IVS  targets  would  be  within  25  cm  of  the  average  positions  derived  from  the 
ongoing  daily  IVS  measurements.  This  approach  gives  an  estimate  of  the  precision  of  the 
positioning  and  is  independent  of  any  ground  truth  errors.  Results  for  the  dynamic  data  responses 
are  detailed  in  Table  6-4.  The  largest  deviation  was  30  cm  for  ISO-2,  which  means  that  this 
objective  was  not  fully  achieved  for  the  dynamic  data  phase  of  the  demonstration. 
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Table  6-4.  TEMTADS  Dynamic  IVS  Positioning  Results 


Seed  Item 

Maximum  Error  (m) 

Average  Error  (m) 

ISO-1 

0.23 

0.15 

ISO-2 

0.30 

0.20 

ISO-3 

0.29 

0.18 

ISO-4 

0.18 

0.10 

0.30 

0.25 

0.20 

0.15 

0.10 

0.05 

0.00 


X 
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Figure  6-1.  Dynamic  IVS  Position  Precision  (Errors  Relative  to  Average  Derived  Positions) 
6.6  Initial  Cued  Survey  Data  Positioning 

This  objective  was  to  demonstrate  positioning  accuracy  during  initial  cued  data  collection  with  the 
TEMTADS.  The  metric  was  assessed  by  evaluating  the  dipole-fit-derived  target  positions  from 
initial  measurements  at  the  IVS.  The  minimum  success  criterion  was  that  100  percent  of  the  fit- 
derived  positions  of  the  IVS  targets  would  be  within  25  cm  of  the  ground  truth  location  for  the 
initial  IVS  measurements.  This  objective  was  achieved  for  the  initial  TVS  cued  survey  (Table  6-5). 
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TABLE  6-5.  TEMTADS  Initial  Cued  IVS  Survey  Positioning  Results 


Seed  Item 

Error (m) 

ISO-1 

0.19 

ISO-2 

0.16 

ISO-3 

0.06 

ISO-4 

0.18 

6.7  Ongoing  Cued  Survey  Data  Positioning 

This  objective  was  to  demonstrate  positioning  precision  during  ongoing  cued  data  collection  with 
the  TEMTADS.  The  metric  was  assessed  by  evaluating  the  dipole-fit-derived  target  positions  from 
daily  measurements  at  the  IVS.  The  minimum  success  criterion  was  that  the  derived  positions  of 
the  IVS  targets  be  within  20  cm  of  the  average  positions  derived  from  the  ongoing  daily  IVS 
measurements.  As  expected,  the  precision  of  the  measurements  was  better  than  the  initial  accuracy 
assessment  because  the  initial  accuracy  assessment  factors  in  ground  truth  measurement  error.  The 
perfonnance  criterion  for  this  metric  was  met  (Table  6-6).  Results  for  the  cued  data  responses  are 
detailed  in  Figure  6-2. 


TABLE  6-6.  TEMTADS  Cued  IVS  Positioning  Results 


Seed  Item 

Maximum  Error  (m) 

Average  Error  (m) 

ISO-1 

0.10 

0.04 

ISO-2 

0.12 

0.05 

ISO-3 

0.12 

0.05 

ISO-4 

0.12 

0.05 
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TOAR  TEMTADS  2x2 

IVS  Results  Offset  Distance  from  Average  Position 
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Figure  6-2.  Cued  IVS  Position  Precision  (Errors  Relative  to  Average  Derived  Positions) 
6.8  Initial  Cued  Measurement  Polarizabilities 

This  objective  was  to  demonstrate  that  the  derived  polarizabilities  of  the  IVS  targets  matched  the 
polarizabilities  in  the  library  during  the  initial  IVS  measurements.  The  minimum  success  criterion 
was  met,  as  the  dipole-fit-derived  polarizabilities  for  the  IVS  targets  during  initial  IVS 
measurements  exhibited  a  decision  metric  of  >0.9  for  all  IVS  ISOs.  These  results  are  provided  as 
the  first  day’s  data  points  on  Figure  6-3. 
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TOAR  TEMTADS  2x2 
IVS  Results  Decision  Metric 
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Figure  6-3.  Decision  Metric  for  IVS  ISOs 

6.9  Ongoing  Cued  Sensor  Polarizability  Precision 

This  objective  was  to  demonstrate  that  the  derived  polarizabilities  of  the  IVS  targets  from  ongoing 
daily  IVS  measurements  matched  the  polarizabilities  derived  from  the  initial  IVS  measurements. 
The  minimum  success  criterion  was  that  the  ongoing  daily  decision  metric  for  the  IVS  targets  was 
better  than  >0.95.  Although  the  initial  IVS  signature  for  each  ISO  can  be  used  for  this  test,  the 
matches  to  the  existing  library  entry  were  sufficient  to  pass  the  MQO,  so  it  was  not  necessary  to 
use  the  initial  IVS-derived  signatures. 

6.10  Cued  Interrogation  Anomalies 

To  collect  data  that  support  classification,  the  source  of  the  anomaly  must  be  illuminated  along  its 
three  principle  axes.  To  ensure  this,  the  sensor  array  must  be  positioned  directly  over  the  center  of 
the  item.  The  metric  for  this  objective  was  to  demonstrate  that  the  center  of  the  array  was  within 
sufficient  distance  of  the  anomaly  source’s  location  during  cued  interrogation.  Positions  of  the 
array  center  were  derived  from  the  RTS  and  IMU  data  along  with  the  derived  target  locations  for 
comparison  against  each  other.  The  array  center  positions  were  compared  against  the  supplied 
target  coordinates  as  part  of  the  daily  QC  process  during  data  collection.  The  minimum  and 
nominal  success  criterion  for  this  objective  was  that  100%  of  the  final  derived  targets  be  positioned 
within  40  cm  of  the  center  of  the  array.  Exceptions  include  targets  that  are  considered  “cannot 
analyze”  (e.g.,  saturated  response  area)  and  multitarget  sources.  Thirty-six  targets  were  classified 
as  “cannot  analyze”  because  the  source  was  too  far  from  the  center  of  the  array  to  support 
classification.  All  of  the  targets  not  considered  “cannot  analyze”  had  a  valid  measurement  within 
this  metric. 
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6.11  Correct  Classification  of  Targets  of  Interest 

Meeting  this  objective  was  a  primary  key  measure  of  the  effectiveness  of  the  classification 
approach.  By  collecting  high-quality  data  and  analyzing  those  data  with  advanced  parameter 
estimation  and  classification  algorithms,  targets  would  be  classified  with  high  efficiency.  The 
metric  for  this  objective  demonstrates  that  TOIs  are  correctly  classified  as  TOIs  on  the  final  ranked 
anomaly  list.  The  ranked  anomaly  list  was  submitted  to  ESTCP  for  scoring  against  the  emplaced 
seed  items  and  the  intrusive  results.  The  minimum  success  criterion  was  correct  classification  of 
100  percent  of  the  seed  items  and  native  TOIs  as  TOIs.  Successful  achievement  of  this  metric 
would  include  seed  items  and  other  TOIs  categorized  as  “dig”  or  “cannot  analyze”  on  the  final 
ranked  anomaly  list.  A  pseudo  receiver  operating  characteristic  (ROC)  curve  presenting  the 
classification  performance  is  shown  in  Figure  6-4.  The  ROC  curve  is  derived  by  moving  down 
the  prioritized  list  and  adding  1  to  the  y  axis  for  each  recovered  TOI  and  1  to  the  x  axis  for  each 
non-TOI  recovery.  A  ROC  curve  representing  perfect  classification  rises  vertically  until  all  TOIs 
are  identified,  then  horizontally  for  the  remaining  non-TOI  results.  A  diagonal  ROC  curve 
indicates  no  classification  performance.  The  results  presented  in  Figure  6-4  indicate  good 
classification  performance.  All  TOIs  were  correctly  categorized  as  “dig”;  six  TOIs  were  initially 
selected  as  Category  -1  (ACDs),  and  the  remaining  26  TOIs  were  all  selected  as  Category  1  (high 
likelihood  TOI). 


Pseudo  ROC  Curve 

- Cat  -1:  ACDs  (training) 

- Cat  0:  Can't  Analyze 

- Cat  1:  High  Likelihood  TOI 

- Cat  2:  Can't  Decide 

- Cat  3:  High  Likelihood  Non-TOI 

0  100  200  300  400  500 

Non-TOI 

Figure  6-4.  Classification  Results  for  Cued  Investigations  of  497  Targets 


6.12  Model  Results  Support  Classification  Decision 

This  objective  was  to  demonstrate  that  data  gathered  exhibit  a  measure  of  the  correlation  between 
the  model  and  the  observed  data,  which  is  used  to  determine  whether  the  model  will  support 
classification.  The  metric  used  to  validate  that  the  model  responses  support  classification  is  the 
percentage  of  targets  that  cannot  be  classified  as  well  as  the  fit  coherence  between  the  model 
response  and  observed  data.  The  targets  that  cannot  be  classified  were  identified  as  having  a  “fit 
coherence”  of  less  than  0.8.  (Fit  coherence  is  an  output  of  the  UXA  dipole  fit  routine  indicating 
the  correlation  of  the  modelled  data  with  the  observed  responses).  The  minimum  success  criterion 
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for  the  number  of  targets  with  responses  that  support  classification  was  that  >90  percent  of  the 
targets  meet  the  fit  coherence  requirement.  Two  targets  were  categorized  as  “can’t  analyze”  due 
to  fit  coherence,  resulting  in  99.6  percent  of  the  targets  meeting  this  metric. 
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7.0  COST  BENEFIT  ANALYSIS 


ESTCP  projects  are  required  to  develop  and  validate,  to  the  extent  possible,  the  expected 
operational  costs  of  the  technology.  The  intent  of  this  section  is  to  identify  the  infonnation  that 
was  tracked  or  the  data  that  were  obtained  during  the  demonstration  that  will  aid  in  establishing 
realistic  costs  for  implementing  the  technology  and  comparing  it  to  potential  alternative 
technologies. 

The  tracked  costs  are  provided  in  Table  7-1,  and  discussion  on  the  cost  elements  provided  in  the 
following  subsections.  Note  that  the  TEMTADS  system  was  provided  by  NRL  at  no  cost  to  CH2M 
for  this  demonstration.  The  costs  per  acre  (for  dynamic  data  collection)  and  per  anomaly  (for  cued 
data  collection)  do  not  include  those  costs. 


Table  7-1.  Costs  for  TEMTADS 


Cost  Element 

Tracked  Data 

Cost/Quantity 

Site  Setup 

Costs  for  planning,  mobilization,  general  site  setup,  shipping  of  equipment,  instrument 
aided  visual  surface  sweep,  QC  seeding,  surveyor  services,  and  demobilization 

$63,000 

MPPEH  management  and  MEC  disposal  for  the  instrument-aided  visual  surface  clearance 
was  de-scoped  from  CH2M  and  passed  along  to  the  removal  action  MEC  contractor 

$5,300 

Dynamic  TEMTADS  Survey  Costs 

Dynamic  detection  survey  (0.89  acre),  including  field  labor  (three 

geophysicists/geophysical  technicians,  one  UXO  technician),  equipment  setup,  equipment 
rentals,  instrument  verification  strip  setup  and  data  collection,  data  processing,  and  per 
diem 

$77,500 
($87, 078/acre) a 

Dynamic  detection  survey  data  processing 

$8,300 

Cued  TEMTADS  Survey  Costs 

Reacquisition  of  anomalies  for  cued  surveys  (497  anomalies),  including  field  labor  (three 
geophysicists/geophysical  technicians,  one  UXO  technician),  equipment  setup,  equipment 
rentals,  instrument  verification  strip  setup,  data  collection,  and  per  diem 

$40,000 

($80.48/anom) 

Processing  of  cued  data,  including  several  site  visits  for  task  kickoff  and  quality  control 
purposes 

$6,400 

($12.88/anom) 

Intrusive  Investigation 

Reacquisition  of  anomalies  and  all  UXO  team-related  costs  related  to  the  intrusive 
investigation  and  documentation  of  discoveries  per  the  ESTCP  intrusive  investigation 
instructions 

$173,200 

($348.49/anom) 

Management  of  material  potentially  presenting  an  explosive  hazard  and  MEC  disposal  for 
the  intrusive  investigation  was  de-scoped  from  CH2M  and  passed  along  to  removal  action 
MEC  contractor 

$11,600 

a  See  implementation  issues  section  for  explanation  of  high  per-acre  cost. 
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8.0  IMPLEMENTATION  ISSUES 


While  the  RTS  can  be  used  for  positioning  in  open  areas  or  narrow  corridors,  the  TEMTADS  unit 
cannot  be  carried  in  such  a  way  that  it  is  always  moving  perpendicular  to  the  RTS  base  station, 
thus  avoid  being  shielded  from  the  base  by  the  operators.  In  order  to  facilitate  positioning  in  a 
heavily  wooded  area  using  RTS,  the  RTS  prism  located  above  the  TEMTADS  unit  had  to  be  raised 
above  the  heads  of  the  operators.  This  allowed  free  movement  by  the  operators  and  increased  the 
data  production  rate;  however,  the  added  height  of  the  prism  more  than  likely  caused  dynamic  IVS 
measurement  MQO  failures  due  to  the  oscillation  of  the  prism  while  walking.  Additionally, 
dynamic  data  with  a  lower  prism  height  were  not  collected,  so  it  is  not  possible  to  rule  out  that  just 
walking  in  litter  mode  with  the  RTS  was  not  the  issue.  Prior  experience  using  the  RTS  with  the 
TEMTADS  in  wheeled  mode  had  no  positioning  MQO  failures. 

Data  collection  production  rates  were  hindered  for  the  following  reasons: 

•  Approximately  1 .5  hours  of  travel  time  was  needed  after  reaching  the  site  each  day  to 
access  the  collection  grids. 

•  Daily  reassembly  and  breakdown  were  required,  as  there  was  no  secure  storage  facility. 

A  temporary  carport  was  installed  for  a  partial  breakdown  area;  however,  cables  needed 
to  be  secured  nightly  due  to  animal  activity. 

•  Setting  up  the  RTS  base  stations  in  multiple  locations  took  more  time  than  a  standard 
RTK  GPS  set  up.  There  was  also  a  steep  learning  curve  for  speedy  assembly  of  the  RTS; 
this  often  delayed  the  start  of  production. 

•  Due  to  the  experimental  nature  of  the  NRL  TEMTADS,  several  days  of  production  were 
lost  due  to  software  issues  and  a  hard  drive  failure,  which  resulted  in  several  trips  to  NRL 
for  solutions. 
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9.0  CONCLUSIONS 


CH2M  performed  an  ESTCP  Munitions  Response  Live  Site  Demonstration  at  MRS  R-04A  West 
at  TOAR  FUDS,  Pennsylvania.  The  demonstration  involved  the  participation  of  two  advanced 
EMI  sensors  and  was  designed  to  investigate  the  use  of  these  advanced  EMI  sensors  at  a  densely 
wooded  site  with  challenging  microterrain  features  (e.g.,  impact  craters,  rocks,  boulders,  gullies) 
for  detection  of  munitions  down  to  the  size  of  37-nnn  projectiles.  The  NRL  TEMTADS  was 
demonstrated  in  both  dynamic  and  cued  modes  in  a  two-person  litter  carry  configuration. 
Collection  and  processing  of  dynamic  and  cued  data  with  an  MPV  was  performed  by  Black  Tusk 
Geophysics,  the  results  of  which  are  addressed  under  separate  cover  by  Black  Tusk  Geophysics. 
Both  instruments  were  coupled  with  a  Trimble  RTS  for  positioning. 

Approximately  0.71  acre  was  dynamically  surveyed  with  the  TEMTADS  system.  Production  was 
significantly  hindered  by  the  remote  location  and  site  conditions,  and  only  one  of  the  four  grids 
initially  selected  for  investigation  achieved  100%  coverage  (not  including  gaps  due  to  physical 
obstructions).  Based  upon  this  effort,  production  rates  under  less  challenging  conditions  are 
estimated  to  be  much  higher. 

A  total  of 429  anomalies  were  identified  by  the  TEMTADS  dynamic  data  analysis.  These  anomaly 
locations  and  an  additional  68  targets  selected  by  the  MPV  survey  were  interrogated  (cued  data 
collection)  and  classified  with  the  TEMTADS  system.  The  cued  data  were  used  to  classify  the 
targets  as  being  a  potential  TOI  (dig)  or  high-likelihood  non-TOI  (do  not  dig).  The  TEMTADS 
successfully  detected  and  classified  all  known  TOIs  (seeds  and  native  TOI). 
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